When you come across a site that executes a JavaScript, open new window function, and if the popup window is without all the usual window features, e.g. back/forward/reload buttons, status bar, etc., Firefox will automatically treat it as a popup and will not open it as a new tab.
However, if you find this to be a nuisance and want to open all new windows in new tabs, type 'about:config' into the Address bar and search for 'browser.link.open_newwindow.restriction'. Now, change the value to 0 or 1. A value of 0 will open all links the way you have Firefox handle new windows in the settings, and a value of 1 will never open any new windows.
